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(54) IMAGE PROCESSING METHOD AND IMAGE PROCESSING APPARATUS 

(57)Abstract: 

PROBLEM TO BE SOLVED: To solve the problem that it is difficult to 
highly accurately perform rectangle cutout processing in the 
environment of much noise. 

SOLUTION: In an image resulting from binarizing an original image, the 
histogram of the number of black pixels in its horizontal and vertical 
directions is created and the histogram is approximated into trapezoid 
to detect apex coordinates indicating a range where a rectangular area 
is present in the binary image. Then, an inclination direction of the 
rectangular area within the present range is detected to highly 
accurately detect the rectangular area out of the binary image. 
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CLAIMS 



[Claim(s)] 
[Claim 1] 

It is the image-processing approach of detecting a rectangle field from an image, 

A subject-copy image is set in the image made binary, and it is the histogram creation process which 

creates the histogram of level and the vertical number of black pixels, 

The rectangle range detection process of detecting the rectangle field's in said binary image existence 
range by carrying out trapezoid approximation of this histogram, 

The inclination direction detection process of detecting the inclination direction of said rectangle field of 
said existence within the limits, 

The image-processing approach characterized by ****(ing). 
[Claim 2] 

In said rectangle range detection process, 

Trapezoid approximation is performed about each of the level and said histogram in a perpendicular 
direction of said binary image, 

Level and the perpendicular direction of said binary image stick, respectively, and the rectangle field's 
existence range is detected. 

The image-processing approach according to claim 1 characterized by things. 
[Claim 3] 

The image-processing approach according to claim 2 characterized by detecting two or more top-most- 
vertices coordinate candidates who receive said rectangle field in said rectangle range detection 
process. 
[Claim 4] 

The image-processing approach according to claim 3 characterized by detecting said top-most-vertices 
coordinate of the approximated trapezoid in said rectangle range detection process as two or more top- 
most-vertices coordinate candidates who receive said rectangle field. 
[Claim 5] 

The image-processing approach according to claim 4 characterized by specifying the coordinate of four 
top-most vertices from two or more top-most-vertices coordinate candidates of said rectangle field in 
said inclination direction detection process. 
[Claim 6] 

In said inclination direction detection process, 

Based on two or more top-most-vertices coordinate candidates who receive said rectangle field, two or 
more inclination directions of this rectangle field are assumed, 

About the rectangle field shown by each of two or more of these inclination directions, counting of the 

number of black pixels which exists in the exterior is carried out, 

each counting — specifying the one inclination direction based on a result, 

The image-processing approach according to claim 5 characterized by specifying the coordinate of said 
four top-most vertices. 
[Claim 7] 
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In said inclination direction detection process, 

Based on two or more top-most-vertices coordinate candidates who receive said rectangle field, two or 
more inclination directions of this rectangle field are assumed. 

About the rectangle field shown by each of two or more of these inclination directions, counting of the 
black pixel which exists in the interior is carried out, 

each counting — specifying the one inclination direction based on a result, 

The image-processing approach according to claim 5 characterized by specifying the coordinate of said 
four top-most vertices. 
[Claim 8] 

Furthermore, the image-processing approach according to claim 5 characterized by having the skew 
amendment process of performing skew amendment, to the rectangle field determined by the coordinate 
of four top-most vertices detected in said inclination direction detection process. 
[Claim 9] 

The image-processing approach according to claim 8 characterized by performing interpolation by the 
bilinear method in said skew amendment process. 
[Claim 10] 

The image-processing approach according to claim 8 characterized by performing interpolation by 
****************** in said skew amendment process. 
[Claim 11] 

The image-processing approach according to claim 1 characterized by assuming that said rectangle field 
is a rectangle in said rectangle range detection process, and carrying out trapezoid approximation of said 
histogram. 
[Claim 12] 

Said rectangle field is the image-processing approach according to claim 1 1 characterized by being the 
field which shows a photograph. 
[Claim 13] 

Furthermore, the image-processing approach according to claim 1 characterized by having a binary 
chemically-modified [ which makes the subject-copy image of a multiple value binary, and obtains said 
binary image ] degree. 
[Claim 14] 

Said subject-copy image is the image-processing approach according to claim 13 characterized by being 
the image with which the photograph of one sheet has been arranged. 
[Claim 15] 

When said subject-copy image is an image arranged in the location which the photograph of two or more 
sheets left to the main scanning direction, 

The image-processing approach according to claim 13 characterized by creating said histogram for 
every binary image which divided and this divided said binary image for said every photograph in said 
histogram creation process in the main scanning direction. 
[Claim 16] 

When said subject-copy image is an image arranged in the location which the photograph of two or more 
sheets left in the direction of vertical scanning, 

The image-processing approach according to claim 13 characterized by creating said histogram for 
every binary image which divided and this divided said binary image in the direction of vertical scanning 
for said every photograph in said histogram creation process. 
[Claim 17] 

Furthermore, it has the labeling process which performs label attachment to said binary image, 
The image-processing approach according to claim 1 characterized by creating the histogram of said 
number of pixels by which label attachment was carried out in said histogram creation process. 
[Claim 18] 
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• <) 

0 Said labeling process, 

The label attachment process which replaces the block of predetermined size [ as / whose number of 

black pixels in the interior is more than a predetermined rate ] with a predetermined label value in said 

binary image about the black pixel connected with the black block detection process detected as a black 

block, and the inside of the detected black block and this black block. 

The image-processing approach according to claim 17 characterized by ****(ing). 

[Claim 19] 

The image-processing approach according to claim 18 characterized by detecting the block with which 
the whole of the interior consists of a black pixel as said black block in said black block detection . 
process. 
[Claim 20] 

Said black block detection process, 

The division process which divides said whole binary image into the block of said predetermined size, 
The judgment process which judges whether it is a black block for said every block, 
The image-processing approach according to claim 18 characterized by ****(ing). 
[Claim 21] 

In said black block detection process. 

The image-processing approach according to claim 18 characterized by detecting a black block by 
shifting the block location of said predetermined size one by one in said binary image. 
[Claim 22] 

The image-processing approach according to claim 18 characterized by detecting the black block of size 
smaller than this predetermined size in said black block detection process when the black book of said 
predetermined size is not detected in said binary image. 
[Claim 23] 

Said label attachment process, 

The criteria label attachment process which replaces criteria Rhine within said black block with said 
label value, 

The up label attachment process which replaces the black pixel connected above said said criteria Rhine 
with said label value, 

The lower label attachment process which replaces the black pixel connected below said said criteria 
Rhine with said label value, 

The image-pro6essing approach according to claim 18 characterized by ****(ing). 
[Claim 24] 

The above-mentioned black block is the image-processing approach according to claim 18 characterized 
by being a rectangle. 
[Claim 25] 

The above-mentioned black block is the image-processing approach according to claim 18 characterized 
by being a polygon. 
[Claim 26] 

The above-mentioned black block is the image-processing approach according to claim 18 characterized 
by the circular thing. 
[Claim 27] 

It is the image processing system which detects a rectangle field from an image, 

A subject-copy image is set in the image made binary, and it is the histogram creation means to create 
the histogram of level and the vertical number of black pixels, 

A rectangle range detection means to detect the rectangle field's in said binary image existence range 
by carrying out trapezoid approximation of this histogram, 

An inclination direction detection means to detect the inclination direction of said rectangle field of said 
existence within the limits, 
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The image processing system characterized by ****(ing). 
[Claim 28] 

[t is the image-processing program which detects a rectangle field from an image, 

A subject-copy image is set in the image made binary, and it is the code of the histogram creation 

process which creates the histogram of level and the vertical number of black pixels, 

The code of the rectangle range detection process of detecting the rectangle field's in said binary image 

existence range by carrying out trapezoid approximation of this histogram, 

The code of the inclination direction detection process of detecting the inclination direction of said 

rectangle field of said existence within the limits, 

The computer-readable program characterized by ****(ing). 

[Claim 29] 

The record medium which recorded the program according to claim 28. 
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DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] 

Especially this invention relates to the image-processing approach and image processing system which 
detect a rectangle image about the image-processing approach and an image processing system, 
[0002] 

[Description of the Prior Art] 

Conventionally, the algorithm for detecting and starting a rectangle field is variously proposed from the 
subject-copy image. For example, in JP,8-237537,A, a profile extract is performed first and the 
technique of searching a rectangle field is indicated by by looking for the part into which a profile 
accomplishes a straight line. 
[0003] 

[Problem(s) to be Solved by the Invention] 

However, according to the algorithm of rectangle logging of the former, it was difficult to detect the . 
straight-line part of a rectangle profile under an environment with many noises, and to associate four 
straight lines and to detect the rectangle of one piece further. Therefore, it is difficult to perform exact 
logging and it was not able to be said that the precision of a logging result was enough. 
[0004] . . 

It is made in order that this invention may solve the problem mentioned above, and it aims at offering 
the image-processing approach and image processing system which enable detection of a rectangle field 
with high precision also under an environment with many noises. 
[0005] 

[Means for Solving the Problem] 
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The image-processing approach of this invention is equipped with the following processes as a way 

method for attaining the above-mentioned purpose. 

[0006] 

Namely, are the image-processing approach of detecting a rectangle field from an image, and it sets in 
the image which made the subject-copy image binary. [ thing / the / level and histogram creation 
process which creates the histogram of the vertical number of black pixels, and carrying out trapezoid 
approximation of this histogram ] It is characterized by having the rectangle range detection process of 
detecting the rectangle field's in said binary image existence range, and the inclination direction 
detection process of detecting the inclination direction of said rectangle field of said existence within 
the limits. 
[0007] 

Furthermore, it has the labeling process which performs label attachment to said binary image, and is 
characterized by creating the histogram of said number of pixels by which label attachment was carried 
out in said histogram creation process. 
[0008] 

[Embodiment of the Invention] 

Hereafter, 1 operation gestalt concerning this invention is explained to a detail with reference to a 

drawing. 

[0009] 

<The 1 st operation gestalt> 

- System configuration 

Drawing 1 is the block diagram showing the configuration of the computer system by which this 
invention is performed. 101 is CPU and controls the whole system. 102 is a keyboard and is used for the 
data input to a system with the mouse of 102a. 103 is a display and consists of CRT, liquid crystal, etc. 
104 is ROM, 105 is RAM, the store of a system is constituted and the data which the program which a 
system performs, and a system use are memorized. 106 is a hard disk drive unit (HDD), 107 is a floppy 
disk drive unit (FDD), and the external storage used for the file system of a system is constituted. 108 is 
a printer. 
[0010] 

109 is network I/F and is connected to LAN or WAN. Data files, such as a computer program and a 
digital image, can be exchanged with other computer systems through I/F109. A program and data are 
recorded on a hard disk 106, or it is developed by direct RAMI 05 and they are performed. 

[0011] 

110 is an image scanner, can read the photograph and printed matter on a manuscript base, and can 
incorporate them on RAMI 05 as digital image data of a multiple value. 

[0012] 

- Rectangle image outline 

An example of the image captured and made binary from the image scanner 110 by drawing 2 is shown. 
The rectangle in drawing shows the manuscript image (a rectangle image is called hereafter) of the 
rectangle of a photograph etc., and three ellipses show the noise. In addition, although the noise was 
simplified and shown in this drawing for explanation, size and the configuration of a part for an actual 
noise are unfixed, and the number also becomes hundreds to about thousands of pieces in many cases. 
Moreover, it supposes binaryHzation of an image that the simple binarization method of common 
knowledge by the comparison with a predetermined threshold is used, and detailed explanation is 
omitted here. 
[0013] 

The image shown in drawing 2 is held on RAMI 05 as a bit map with a width of face [ of W pixels ], and a 
height of H pixels. Each pixel has a depth of 1 byte and a black pixel presupposes that a value 255 and a 
white pixel are shown by the value 0. Moreover, the bit map on RAMI 05 shall be accessed as two- 
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dimensional array, and writes buf [ the j-th pixel ] (i, j) for the upper left of this bit map to the i-th 
perpendicular direction horizontally as an origin. With this operation gestalt, the coordinate of the 
existence region of a rectangle image is specified by the approach shown below from this bit map. 
[0014] 

- Histogram creation 

Drawing 3 is drawing showing the situation of the histogram which carried out counting of the black pixel 
horizontally when the ideal condition that a noise does not exist in the binary image shown in drawing 2 
is assumed, this drawing upper part shows a binary image, and the lower part shows the example of a 
histogram. According to this drawing, corresponding to four top-most vertices of the rectangle of a 
binary image, it turns out that a histogram makes a trapezoid, in general. In addition, although a histogram 
does not serve as a perfect trapezoid since an actual binary image contains a noise, sufficient 
information to pinpoint the location of the top-most vertices which can be set horizontally can be 
acquired from the trapezoid configuration of this histogram. 
[0015] 

Drawing 4 is a flow chart which shows creation processing of the histogram shown in drawing 3 . 

Hereafter, histO shows the array which stores histogram data. 

[0016] 

First, in steps S401 and S402, a value 0 is assigned to Variable x and each which y Reaches, and a value 
0 is assigned to Variable hist (x) at step S403. 
[0017] ' 

And in step S404, Pixel buf (x y) and the value 255 on a bit map are compared, if it differs, it will 
progress to step S406, but if equal, after progressing to step S405 and only a value's 1 incrementing hist 
(x), it progresses to step S406. 
[0018] 

At step S406, only a value 1 increments Variable y and y is compared with height H of a bit map in step 
S407 below. If y differs from H, it will return to step S404, but if equal, it will progress to step S408, only 
a value 1 will increment Variable x, and step S409 will compare x and the bit map width of face W. If x 
differs from W, it will return to step S402, but processing will be ended if equal. 
[0019] 

- Top-most-vertices detection processing 

By the above processing, array histO of the histogram which carried out counting of the black pixel is 
obtained. This histogram is making the trapezoid surrounded in the X-axis and three straight lines, as 
shown in drawing 3 . With this operation gestalt, the location at which c and the right-hand side cross 
the X-axis in the location at which b, the right-hand side, and a raised bottom cross the location at 
which a, left part, and a raised bottom cross the location at which left part crosses the X-axis is set to 
d in this trapezoid. 
[0020] 

The trapezoid left part which a histogram presents here using the flow chart of drawing 5 is analyzed, 

and how to ask for the above-mentioned locations a and b is explained. 

[0021] 

In step S501, the peak point P that hist (x) becomes max is detected first. In the histogram shown in 
drawing 3 , the peak of height 800 is in the location before and behind X= 400. Next, in step S502, the 
point F which becomes the height of the one half of a peak at the front of Point P is detected. In the 
histogram shown in drawing 3 , the location before and behind X= 100 is equivalent to Point F. And at 
step S503, the point F0 which becomes. one fourth of the height of a peak at the front of Point F is 
detected, and the point F1 which becomes three fourths of the height of a peak at step S504 at the 
back of Point F is detected further. 
[0022] 

Next, in step S505, the straight line L as shown in drawing 3 is obtained by performing straight-line 
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approximation about all the points of FO <=X<=F1 using well-known regression analysis. And the point 
that a straight line L crosses the X-axis in step S506 is computed, and this point is acquired as a. And 
in step S507, the point that a straight line L becomes 98% of the height of Peak P is computed, and this 
point is acquired as b. This is for amending this 98% at a minute in consideration of having come out 
[ more ] than an actual peak, since Peak P contains a part for a noise. As mentioned above, although 
the example which performs retrieval to the front of Peak P and acquires Points a and b with the flow 
chart shown in drawing 5 was explained, Points c and d can be acquired by searching to the back of 
Peak P similarly. 
[0023] 

Moreover, although the above is explanation about horizontal top-most-vertices location detection, it is 
possible by reversing the axis of ordinate and axis of abscissa in the above-mentioned explanation, and 
completely applying the same approach to perform top-most-vertices location detection in a 
perpendicular direction. As this shows drawing 6 , the coordinate value of each four every direction 
surrounding a rectangle, i.e., eight variables, (ax, bx, cx, dx, ay, by, cy, dy) can be obtained as a top- 
most-vertices candidate. 
[0024] 

Here, although the eight above-mentioned variables are determined as a meaning from a rectangle, a 
rectangle cannot be conversely determined from eight variables. That is, although four kinds of top- 
most-vertices coordinates can be considered according to the direction of a skew (henceforth, skew 
mode) as a rectangle based on eight variables as shown in drawing 7 , it cannot be determined the any 
they are. In addition, in drawing 7 , a rectangle sets four skew modes to the modes 1 and 2, -1, and -2, 
respectively by making into the mode 0 the completely level case (un-illustrating) of being special. From 
the image shown in each [ these ] mode, since the histogram of the same configuration is generated, it 
is necessary to judge skew mode by other approaches other than a histogram. 
[0025] 

- Skew mode judging 

Hereafter, the skew mode judging processing in this operation gestalt is explained to a detail. 
[0026] 

although drawing 6 mentioned above is equivalent to the condition in the mode 1 shown in drawing 7 , if 
counting of the black pixel which exists inside two angular domains [ three ] shown with the slash in 
drawing 6 is carried out, since a black pixel (halftone dot part in drawing) is not contained ideally — 
counting of a black pixel — it is thought that a result is set to 0. Hereafter, the slash field. shown in this 
drawing 6 is called the template in the mode 1. In addition, let similarly two angular domains [ three ] 
which exist in the rectangle upper part be the templates in each mode about other modes shown in 
drawing 7 . 
[0027] 

if the template (shadow area in drawing) in the mode 2 is applied to the rectangle image (halftone dot 
part in drawing) which is the mode 1 in fact as shown in drawing 8 here, since a rectangle part will 
overflow inside a template — counting of the number of black pixels — a result is not set to 0. With this 
operation gestalt, rectangular skew mode is judged using this property. 
[0028] 

the number of black pixels at the time of specifically applying the template in the four modes to the 
rectangle image used as the candidate for a judgment — respectively — counting — carrying out — 
counting — the template whose result is the minimum is determined as skew mode of the image. That is, 
the case where there are few entering degrees to a template presupposes that it is the skew degree 
which this rectangle image approximates most, of course, the black pixel which exists out of a template 

— counting — carrying out — counting — it is also possible to determine skew mode by the template 
whose result is max. 

[0029] 
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By the above processing, four top-most-vertices coordinates of the rectangle field in a binary image 
can be specified. Therefore, based on this information, the photograph, of the rectangle arranged for 
example; on an image scanner 110 can be cut down by performing well-known skew amendment and 
image logging processing to a subject-copy image. 
[0030] 

In addition, what is necessary is just to use properly according to an application, since it is desirable as 
skew amendment processing of the above-mentioned common knowledge to use simple interpolation 
(******************) or a Hnear interpolation method (bilinear law), the former is excellent in a rate 
and the latter is excellent in image quality. 
[0031] 

A rectangle field is correctly detectable even if it is under an environment with many noises according 
to this operation gestalt, as explained above. 
[0032] - 

In addition, although the ideal trapezoid from which an oblique side on either side becomes symmetrical 
as an ideal histogram should be presented if a rectangle field is a perfect rectangle, an oblique side on 
either side may not serve as symmetry under the effect of a noise, namely, the distance between a-b 
shown in drawing 3 may not become equal to the distance between c-d. In such a case, trapezoid 
approximation is performed in accordance with the data obtained from the oblique side of the direction 
considered that there is little effect of a noise. What is necessary is to specifically measure the 
precision (square error between a straight line and live data) of the straight-line approximation by 
regression analysis about an oblique side on either side, and just to adopt the way with few errors. 
[0033] 

It is also considered that the rectangle which similarly connected four top-most vertices obtained as a 
result of rectangle detection does not necessarily turn into a rectangle under the effect of a noise, and 
the adjacent side does not intersect perpendicularly. Also in such a case, the precision of rectangle 
detection can be improved by correcting based on the data of the direction considered that there is 
little effect of a noise, so that a rectangle may turn into a rectangle. 
[0034] 

<The 2nd operation gestalt> 

Hereafter, the 2nd operation gestalt concerning this invention is explained. In addition, since the system 
configuration in the 2nd operation gestalt is the same as that, of the 1st operation gestalt mentioned 
above, explanation is omitted. 
[0035] 

In the 1st operation gestalt mentioned above, the case where, one rectangle field existed in an image 
was explained. It scans, where it was not necessarily level and one photograph is laid on the manuscript 
base of an image scanner 110 as such a situation, and it is performing skew amendment and image 
logging automatically, after that, and the processing which extracts a photograph is assumed. 
[0036] : 

However, since the manuscript base of a common flat head scanner enables installation of the 
manuscript of A4 size extent, it can lay two or more [ of common photographs (the so-called L-th 
edition size) ] at once. Therefore, if only the rectangle vfield which reads such a photograph of two or 
more sheets with one scan, and shows a photograph from the obtained image can be started, the 
increase in efficiency of a scan can be attained. 
[0037] 

Then, in the 2nd operation gestalt, it is characterized by detecting two or more rectangle images from 

the image of one sheet. 

[0038] 

Hereafter, with reference to drawing 9 , detection processing of two or more rectangle images which can 
be set in the 2nd operation gestalt is explained. The left part of drawing 9 shows the binary-ized image 
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at the time of having detached and arranged two photographs in the direction of vertical scanning of a 
manuscript base (longitudinal direction in drawing), and the right part of drawing 9 shows the situation of 
the histogram in the direction of vertical scanning of this binary-ized image. As shown in this drawing, 
when two or more photographs separate and are arranged, two or more trapezoids appear in a histogram. 
Therefore, by referring to this histogram in the 2nd operation gestalt, it is the location shown by 
dashed-line L in between trapezoids (i.e., drawing 9 ) of these plurality, and an image is carved into two. 
Then, in each field kicked by this end, a rectangle image is detectable henceforth by the approach 
shown with the 1st operation gestalt mentioned above, respectively. 
[0039] 

In addition, although the case where two or more photographs had been detached and arranged in the 
direction of vertical scanning in drawing 9 was explained, when this has been detached and arranged to 
the main scanning direction (the ******** direction), similarly, by creating the histogram about a main 
scanning direction, an image can be carved in a main scanning direction and each rectangle image can 
be extracted. 
[0040] 

As explained above, according to the 2nd operation gestalt, two or more rectangle images are detectable 

from the image of one sheet. 

[0041] 

<The 3rd operation gestalt> 

Hereafter, the 3rd operation gestalt concerning this invention is explained. In addition, since the system 
configuration in the 3rd operation gestalt is the same as that of the 1st operation gestalt mentioned 
above, explanation is omitted. 
[0042] 

In the 3rd operation gestalt, by performing ideal labeling processing to a binary image as shown in 
drawing 2 , the effect of a noise is eliminated and it is characterized by detecting a rectangle image to 
high degree of accuracy more. 
[0043] 

Here, the ideal labeling processing in the binary image shown in drawing 2 is giving the pixel value 1 to all 
the pixels contained in an upper rectangle object based on connection of black pixels, and giving the 
pixel value 2 to all the pixels contained in a downward rectangle object. That is, a label is not given to a 
part for a minute noise, but it leaves to the pixel value 255. 
[0044] 

- Labeling processing outline 

the following and the above — the labeling approach for realizing ideal labeling is explained to a detail. 
[0045] 

Drawing 10 is drawing for explaining the retrieval approach of the block which corresponds to the upper 
left part containing the upper rectangle object in drawing 2 , and serves as an origin of the continuous 
tone processing in the 3rd operation gestalt. 
[0046] 

As first shown in drawing 10 , the W pixel xH pixel whole bit map shown in drawing 2 is divided into the 
block of w pixel xh pixel, and it searches for a block ("a black block" is called hereafter) with which the 
whole of the interior is occupied by the black pixel of a value 255 in the drawing Nakaya mark direction. 
The block shown with the black rectangle in drawing 3 is a found black block. 
[0047] 

With the 3rd operation gestalt, high-speed labeling is realized by smearing away the field which the black 
pixel has connected with the found black block. The flow chart of the whole labeling processing in the 
3rd operation gestalt is shown and explained to drawing 1 1 . 
[0048] 

First, at step S1401, a value 1 is assigned to Variable c, and black block retrieval is performed at step 
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S1402. About the detail of this black block retrieval processing, it mentions later. 
[0049] 

Next, it judges whether the black block was found in step S1403, if found, it will progress to step S1404, 

but processing is ended if not found. 

[0050] 

In step S1404, a black pixel connection field is smeared away with Label c with the found black block as 

the starting point. About the detail of this continuous tone processing, it mentions later. 

[0051] 

And only 1 increments Variable c at step S1405, and return and the above processing are repeated to 

step S1402. 

[0052] 

Labels 1 and 2, — , c can be given only to an object with the magnitude more than predetermined as a 

result of this processing. 

[0053] 

- Black block retrieval processing 

Hereafter, the detail of the black block retrieval processing shown in step S1402 of drawing 1 1 is shown 

and explained to the flow chart of drawing 12 . 

[0054] 

In steps S1501-S1504, Variables yx, j, and i are first initialized with a value 0, respectively. And buf (x+i, 
y+j) judges whether it is 255 which shows a black pixel at step S1505, and if it is 255, and it is not 255, it 
will progress to step S1506 to step S1510. 
[0055] 

At step S1506, if equal to w with which only a value 1 increments Variable i and i indicates block width 
of face to be at step S1507, it progresses to step S1508, and if it differs, it will progress to step S1505. 
[0056] 

At step SI 508, if equal to h with which only a value 1 increments Variable j and j indicates block height 
to be at step S1 509, processing will be ended, and at this time, it is detected noting that the wxh pixel 
block which makes (x, y) an upper left origin is a black block. When j differs from h, it returns to step 
S1504. 
[0057] 

At step S1510, if it is more than W with which add w to x and x indicates image width to be at step 

S1 51 1, it will progress to step S1 51 2, otherwise, will return to step S1 503. 

[0058] 

At step S1512, h is added to y, if it is more than H y indicates image height to be at step S1513, 
processing will be ended, and it is judged with a black block not existing at this time. With [ y ] H 
[ under ], it returns to step S1502. 
[0059] 

The black block of w pixel xh pixel is detected from a W pixel xH pixel binary image by the above 

processing. 

[0060] 

- Continuous tone processing of a connection field - 

Hereafter, the detail of continuous tone processing of the connection field shown in step S1404 of 

drawing 1 1 is shown and explained to the flow chart of drawing 13 . 

[0061] 

In addition, various approaches are learned as continuous tone processing of an object, and what kind of 
approach of smearing away may be used with the 3rd operation gestalt. Therefore, it supposes that a 
fundamental approach is explained here and an object assumes outside that it is the simple configuration 
which is a convex for simplification of explanation. 
[0062] 
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it is shown in drawing 1 4 as a result of the black block retrieval processing mentioned above — as (x y) 

- the block made into an upper left origin is detected as a black block, then, continuous tone 
processing of the 3rd operation gestalt — setting — the central point of this black block — it is (/2, 
(y+h)/2) — (x+w) it considers as an origin and the field connected by the black pixel is smeared away 
with Label c. That is, the pixel value which is a value 255 is replaced with a value c. 

[0063] 

Specifically, the black joining segment on this scanning line is first smeared away with Label c in step 
S601 about the scanning line including the central point (x+w) (/2, (y+h)/2) of a black block. That is, the 
segment (segment which cuts an object horizontally in drawing 14 ) which connects the left end (xO, 
y+h/2) and right end (x1, y+h/2) of the above-mentioned scanning line in an object including the black 
block shown in drawing 14 is smeared away with Label c. In addition, below, the segment which connects 
(x1, y+h/2) to the above (x0 f y+h/2) is called a connection segment, and is later mentioned about the 
detail of the continuous tone processing. 
[0064] 

And the black joining segment which is upwards from a connection segment at continuous tone and step 
S603 about the black joining segment which is downward from a connection segment at step S602 is 
smeared away. In addition, about the detail of the continuous tone processing in steps S602 and S603, it 
mentions later. 
[0065] 

The field connected with the black block by the black pixel can be smeared away with Label c as a 

result of this processing. 

[0066] 

- Continuous tone processing of a connection segment 

Hereafter, the detail of continuous tone processing of the connection segment shown in step S601 of 
drawing 13 is shown and explained to the flow chart of drawing 15 . In addition, suppose that a 
coordinate (X, Y) shows the central point (x+w) (/2, (y+h)/2) of the black block which serves as an origin 
of processing for simplification of explanation. That is, it is referred to as X= (x+w)/2, and Y= (y+h) / 2. 
[0067] 

First, X is substituted for Variable i at step S801, and only a value 1 carries out the decrement of the i 
at step S802. And if it judges and differs [ whether buf (i, Y) is equal to a value 255, and ] at step S803, 
it will progress to step S804, but if equal, it will return to step S802 and the decrement of the i will be 
carried out further. 
[0068] 

A value i+1 is stored in a variable xO at step S804, and X is substituted for Variable i at step S805. And 
only a value 1 increments i at step S806, and it judges whether buf (i, Y) is equal to a value 255 at step 
S807, if equal, it will progress to step S808, but if it differs, it will return to step S806 and i will be 
incremented further. 
[0069] 

A value M is stored in a variable x1 at step S808, a value xO is assigned to Variable i at step S809, and 
c is substituted for step S810 at buf (i, Y). And although only a value 1 increments i at step S81 1, and 
processing will be ended at step S812 if i is larger than xt, it returns to step S810. 
[0070] 

By this processing, the x-coordinate in the both ends of a connection segment can be stored in 

variables xO and x1, and this connection segment can be smeared away with Label c. 

[0071] 

- Continuous tone processing which is the bottom (above) from a connection segment 

Hereafter, the detail of continuous tone processing of the black joining segment which is downward is 
shown and explained to the flow chart of drawing 16 from the connection segment shown in step S602 
of drawing 13 . In addition, the black joining segment below the connection segment which sets to 
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Y=y+h/2 also here, therefore connects (xO, Y) f and (x1, Y) is smeared away with Label c. 
[0072] 

First, only a value 1 increments Variable Y at step S901, and xO is substituted for Variable i at step 
S902. And if equal [ it judges and differs / whether buf (i, Y) is equal to a value 255 and / at step S903, 
and ], it will progress to step S904 to step S906. 
[0073] 

Although only a value 1 increments i, and processing will be ended at step S905 in step S904 if i is 

larger than x1, it returns to step S903. 

[0074] 

In step S906, i is substituted for X and the connection segment on a longitudinal direction, i.e., the 
scanning line, is smeared away with (X, Y as the starting point) at step S907. In addition, since the flow 
chart of drawing 15 showed the continuous tone processing on this scanning line, that detailed 
explanation is omitted here. 
[0075] 

In addition, what is necessary is to be the same as that of the continuous tone processing to down 
[ which is shown in the flow chart of drawing 16 fundamentally mentioned above], and just to perform 
[ about continuous tone processing of the black joining segment which is upwards from the connection 
segment shown in step S603 of drawing 13 , / steps / other ] same processing that what is necessary is 
just to carry out a decrement instead of incrementing Y in step S901. 
[0076] 

According to labeling processing which was explained above, label attachment can be performed about 
an object including the black block of predetermined size, i.e., an object with the area more than 
predetermined size. Therefore, a label is not attached to a minute noise field and high-speed labeling 
becomes possible. 
[0077] 

- Rectangle detection after labeling 

To all the pixels by which the pixel value 1 as a label 1 is included in a downward rectangle object by the 
above labeling processing in the two-dimensional array of buf() shown in drawing 2 to all the pixels 
contained in the rectangle object of this drawing upper part, the pixel value 255 is stored in the black 
pixel of the minute noise part with the other pixel value 2 as a label 2, and a value 0 is stored in the 
white pixel of a background, respectively. 
[0078] 

In the 3rd operation gestalt, the location (top-most-vertices coordinate) of a rectangle object is judged 
for every label field. That is, rectangle detection processing in which it explained in the 1st and 2nd 
operation gestalt mentioned above is performed to the image after labeling. What is necessary is just to 
change into a label value (1 or 2) the candidate for a comparison of bufO shown in step S404 from the 
value 255 which shows a black pixel in the creation flow chart of the histogram specifically shown in. 
drawing 4 mentioned above. [ in this case ] Thereby, the field of a label 1 and a label 2 is detected as a 
rectangle. 
[0079] — 

In addition, although the case where the rectangle of two pieces was included here was explained as an 
example, it cannot be overemphasized that it is possible to perform the above-mentioned labeling , 
processing to the binary image containing the rectangle of the arbitration number, and to perform 
rectangle detection processing for every label. 
[0080] 

As explained above, according to the 3rd operation gestalt, by performing ideal labeling processing to a 
binary image, the effect of a noise can be eliminated and a rectangle image can be detected more to 
high degree of accuracy. 
[0081] . 
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<The 4th operation gestalt> 

Hereafter, the 4th operation gestalt concerning this invention is explained. In addition, since the system 
configuration in the 4th operation gestalt is the same as that of the 1st operation gestalt mentioned 
above, explanation is omitted. 
[0082] 

In the 3rd operation gestalt mentioned above, when searching for a black block from a binary image, 
after dividing the whole binary image into a block first, it was confirmed whether all the interior of each 
block was black pixels. However, other approaches can be applied as the retrieval approach of a black 
block, for example, it sets in the 4th operation gestalt, and how to change continuously the upper left 
coordinate of the block in a binary image is explained. Although an operation load becomes high 
according to this approach, the detectability ability of a black block can be improved. 
[0083] 

Drawing 17 is drawing showing signs that it searches for the black block in a rectangle object in the 4th 
operation gestalt. According to this drawing, from the block boundary which showed the origin (x y) of 
the retrieval in the 4th operation gestalt to drawing 10 with the 3rd operation gestalt, it turnsout that it 
is in the location from which it separated. 
[0084] 

The flow chart of retrieval processing of the black block in the 4th operation gestalt is shown in drawing 

18 , and it explains to it at a detail. 

[0085] 

In steps S1 101-S1 104, Variables yx, j, and i are first initialized with a value 0, respectively. And buf (x+i, 
y+j) judges whether it is 255 which shows a black pixel at step S1105, and if it is 255, and it.is not 255, it 
will progress to step S1 106 to step S1 1 10. 
[0086] 

At step S1 106, if equal to w with which only a value 1 increments Variable i and i indicates block width 
of face to be at step 31 107, it progresses to step S1 108, and if it differs, it will progress to step S1 105. 
[0087] 

At step S1108, if equal to h with which only a value 1 increments Variable j and j indicates block height 
to be at step S1109, processing will be ended, and at this time, it is detected noting that the.wxh pixel 
block which makes (x, y) an upper left origin is a black block. When j differs from h, it returns to step 
S1104. 
[0088] 

At step S1 1 10, i+1 is added to x. In addition, although the same result is final completely obtained even 
if only a value 1 increments x here, the futility of retrieval is excluded by adding i+1, and processing can 
be accelerated. 
[0089] 

And if it is more than W x indicates image width to be at step S1 1 1 1, it will progress to step S1 1 12, 

otherwise, will return to step S1103. 

[0090] 

At step S1 1 12, only a value 1 increments y, if equal to H y indicates image height to be at step S1 1 13, 
processing will be ended, and it is judged with a black block not existing at this time. If y is not equal to 
H, it returns to step S1 102. 
[0091] 

The black block of w pixel xh pixel is detected from a W pixel xH pixel binary image by the above 
processing. And continuous tone of a black pixel connection field and rectangle detection processing* 
can be performed like the 3rd operation gestalt mentioned above henceforth. 
[0092] 

As explained above, according to the 4th operation gestalt, the black block of predetermined size is 
more detectable to high degree of accuracy. Therefore, labeling only about an effective object can be 
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carried out more to high degree of accuracy, as a result highly precise rectangle detection is attained. 
[0093] 

In addition, according to the 3rd and 4th operation gestalt mentioned above, it made for all the pixels 
within a block to be black pixels into the criteria of a black block, but many white pixels may mix in the 
interior of a black field, for example depending on the condition of a noise. Then, in this invention, the 
number of black pixels inside a block is counted, and when this exceeds a predetermined threshold, you 
may judge with it being a black block. 
[0094] 

Moreover, the black block for which it should search does not necessarily need to be a rectangle, for 
example, according to the configuration of polygons, such as a hexagon, and the object which may be 
circular or an ellipse form and is assumed, it is selectable, and it is good to also make. 
[0095] 

Moreover, if the size of a retrieval block also begins not only one predetermined size but a block size big, 
for example to retrieval and an object does not find it, size is gradually made small and you may make it 
repeat retrieval. Thereby, it is assumed and the danger that reliance will also take a small object for a 
noise and will disregard it can be avoided. 
[0096] 

[Other operation gestalten] • 

In addition, even if it applies this invention to the system which- consists of two or more devices (for 
example, a host computer, an interface device, a reader, a printer, etc.), it may be applied to the 
equipments (for example, a copying machine, facsimile apparatus, etc.) which consist of one device. 
[0097] 

Moreover, it cannot be overemphasized that it is attained also by reading and performing the program 
code with which the purpose of this invention supplied the storage which recorded the program code of 
the software which realizes the function of the operation gestalt mentioned above to a system or 
equipment, and the computer (or CPU or MPU) of the system or equipment was stored in the storage. 
[0098] 

In this case, the function of the operation gestalt which the program code itself read from the storage 
mentioned above will be realized, and the storage which memorized that program code will constitute 
this invention. 
[0099] 

As a storage for supplying a program code, a floppy disk, a hard disk, an optical disk, a magneto-qptic . 
disk, CD-ROM, CD-R, a magnetic tape, the memory card of a nop-volatile, ROM, etc. can be used, for 
example. 
[0100] 

Moreover, it cannot be overemphasized that it is contained also when the function of the operation 
gestalt which performed a part of processing that OS (operating system) which is working on a 
computer was actual, based on directions of the program code, and the function of the operation gestalt 
mentioned above by performing the program code which the computer read is not only realized, but was 
mentioned above by the processing is realized. 
[0101] 

Furthermore, after the program code read from a storage is written in the memory with which the 
functional expansion unit connected to the functional add-in board inserted in the computer or a 
computer is equipped, it cannot be overemphasized that it is contained also when the function of the 
operation gestalt which performed a part or all of processing that CPU with which the functional add-in 
board and functional expansion unit are equipped based on directions of the program code is actual, and 
mentioned above by the processing is realized. 
[0102] 

[Effect of the Invention] 



- 15- 



J 

As explained above, according to this invention, a rectangle field is detectable with high precision also 
under an environment with many noises. 
[Brief Description of the Drawings] 

[Drawing 1] It is the block diagram showing the outline configuration of the image processing system in 1 
operation gestalt concerning this invention. 

[Drawing 2] It is drawing showing the example of a binary image which serves as a candidate for 
detection in this operation gestalt. 

[Drawing 3] It is drawing showing the example of a horizontal histogram of the number of black pixels in 
this operation gestalt. 

[Drawing 4] It is the flow chart which shows the histogram calculation processing in this operation 
gestalt. 

[Drawing 5] It is the flow chart which shows the trapezoid approximation approach of the histogram in 
this operation gestalt. 

[Drawing 6] It is drawing showing the example of a detection coordinate of the rectangle image in this 
operation gestalt. 

[Drawing 7] It is drawing showing four skew modes in this operation gestalt. 

[Drawing 8] It is drawing for explaining the principle of the skew mode judging in this operation gestalt. 
[Drawing 9] In the 2nd operation gestalt, it is drawing showing the situation of the histogram at the time 
of having arranged two or more photographs. 

[Drawing 10] It is drawing for explaining the black block retrieval approach in the 3rd operation gestalt. 
[Drawing 11] It is the flow chart which shows the outline of the labeling processing in the 3rd operation 
gestalt. 

[Drawing 1 2] It is the flow chart which shows the black block retrieval processing in the 3rd operation 
gestalt. 

[Drawing 13] It is the flow chart which shows the continuous tone processing in the 3rd operation 
gestalt. 

[Drawing 14] It is drawing for explaining the connection segment continuous tone approach in the 3rd 
operation gestalt. 

[Drawing 15] It is the flow chart which shows the connection segment continuous tone processing in the 
3rd operation gestalt. 

[Drawing 16] It is the flow chart which shows continuous tone processing of the bottom field of a 
connection segment in the 3rd operation gestalt. 

[Drawing 1 7] It is drawing for explaining the black block retrieval approach in the 4th operation gestalt. 
[Drawing 18] It is the flow chart which shows the black block retrieval processing in the 4th operation 
gestalt. 

[Description of Notations] 

101 CPU 

102 Keyboard 
102a Mouse 

103 Display 

104 ROM 

105 RAM 

106 Hard Disk 

107 Floppy Disk 

108 Printer 

109 Network I/F 

110 Image Scanner 
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* NOTICES* 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

"I.This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 

DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] It is the block diagram showing the outline configuration of the image processing system in 1 
operation gestalt concerning this invention. 

[Drawing 2] It is drawing showing the example of a binary image which serves as a candidate for 
detection in this operation gestalt. 

[Drawing 3] It is drawing showing the example of a horizontal histogram of the number of black pixels in 
this operation gestalt. 

[Drawing 4] It is the flow chart which shows the histogram calculation processing in this operation 
gestalt. 

[Drawing 5] It is the flow chart which shows the trapezoid approximation approach of the histogram in 
this operation gestalt. 

[Drawing 6] It is drawing showing the example of a detection coordinate of the rectangle image in this 
operation gestalt. 

[Drawing 7] It is drawing showing four skew modes in this operation gestalt. 

[Drawing 8] It is drawing for explaining the principle of the skew mode judging in this operation gestalt. 
[Drawing 9] In the 2nd operation gestalt, it is drawing showing the situation of the histogram at the time 
of having arranged two or more photographs. 

[Drawing 10] It is drawing for explaining the black block retrieval approach in the 3rd operation gestalt. 
[Drawing 1 1] It is the flow chart which shows the outline of the labeling processing in the 3rd operation 
gestalt. 

[Drawing 12] It is the flow chart which shows the black block retrieval processing in the 3rd operation 
gestalt. 

[Drawing 13] It is the flow chart which shows the continuous tone processing in the 3rd operation 
gestalt. 

[Drawing 14] It is drawing for explaining the connection segment continuous tone approach in the 3rd 
operation gestalt. 

[Drawing 1 5] It is the flow chart which shows the connection segment continuous tone processing in the 
3rd operation gestalt. 

[Drawing 16] It is the flow chart which shows continuous tone processing of the bottom field of a 
connection segment in the 3rd operation gestalt. 

[Drawing 17] It is drawing for explaining the black block retrieval approach in the 4th operation gestalt. 
[Drawing 18] It is the flow chart which shows the black block retrieval processing in the 4th operation 
gestalt. 

[Description of Notations] 

101 CPU 

1 02 Keyboard 
102a Mouse 

103 Display 

104 ROM 
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105 RAM 

106 Hard Disk 

107 Floppy Disk 

108 Printer 

109 Network l/F 

110 Image Scanner 
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